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SECURITY APPARATUS AND METHOD Further in accordance with the present invention, there is 

provided a security method associated with supplying a 
BACKGROUND OF THE INVENTION service to a user* comprising the si.ep of: recognizing a user 

1. Field of the Invention desiring the service: supplying the service to the user if the 
The present invention relates to a security apparatus and 5 uscr is ^cognized; deciding whether the user is under a 

a method for supplying a service to a user by confirming a situation to use the service; deciding whether a non-user 

use situation of the user and infringement situation of illtrudes int0 a use *«* of the se ™ ce '> and controlling the 

non-user in order to keep safety and reliability for the ^PP^ of ^ service # the user is ft0t undef * e situation to 

service usc * e serv * ce OT *f the non-user is intruding into the use 

2. Description of the Related Art 20 «■ of fte 

A method to specify the user in service-use area by Further^ accordance with the Resent invention, there is 

electrical means and to execute security for each kind of 3180 P rovided a computer readible memory containing 

service is very important in present electrical society. Vari- computer-readable instructions to supply a service to a user, 

ous lands of security means, for example, memorized < c0 *P™ing: instruction means for causing a computer to 

number, magnetic card. IC card wireless card, sound 15 cognize a user desiring the service; instruction means for 

comparison, fingerprint comparison, retina comparison and causing a computer to supply the service to the user if the 

face comparison, have been used. However, the memorized user is recognized; in'struction means for causing a computer 

number, the magnetic card, the IC card and the wireless card to dccide whether the user is un der a situation to use the 

specifies the user by information only the user knows or with service; to^ction means for causing a computer to decide 

a memory device key for only the user Therefore, they are 20 whether a non-user intrudes into a use area of the service; 

in danger of being used unjustly and instruction means for causing a computer to control a 

On the other hand, sound comparison, fingerprint S !P* of / hC tf &C ™* is dedded r? nder 

comparison, retina comparison, and face comparison are f*"*° D 0 " se *" ™F * no * user 15 decxded t0 

effective personal confirmation means to defecT an other's 25 1Dtrade mt0 use Mta of * e s,;mce ' 

unjust use because the user's physical features are used as a BRIEF DESCRIPTION OF THE DRAWINGS 
key. However, the personal confirmation is only used at the 

begining of service. After the service begins, the service can FIG. I is a block diagram of the security apparatus 

not be interrupted. Therfore. even if an other person changes according to the present invention, 

for the user after the service is begun, the service can not be 30 FIG. 2 is a block diagram of the situation detection section 

interrupted. In short, this kind of service includes danger that 0 f the security apparatus accordic g to the preset in venuon. 

the other person utilizes the service unjusty. This kind of , it . „ flA „, . _ - . . c 

- - , . , . • j t *l FiG. 3 is a flow chart of processing of a service supply 

problem is included in the memorized number, the magnetic ^^a$JL *~ - *Jl ~ u j» . JTi_ 

card, the IC card and the wireless card. permission section according to a first embodiment of the 

present invention. 

As mentiored above, the known security apparatuses 35 rrr n A - „ * . ^ * *^ 

include following problems. FIG. 4 is a flow chart of processing of the service supply 

*~ . . „ , . ^ rxirmssionsecUonaccordmg to a second embodiment of the 

(1) After security is cancelled by confirming the user, present invention 
security function is not executed while supplying the ser- - . ~ 

vice. Therefore, even if an other person changes for the user, H ?' S ^ a flow chart of processing of the service supply 

the service is still supplied con&uously. 40 P^* 10 * sectl0n according to a third embodiment of the 

/">v wtvm 1 . . . , . present invention. 

(2) While supplying the service to the user, it often „ . . . 

happens that an other person watches information not to be n 9' ? 1S a flow cbalt of P rocesSifl g of ^ service supply 

disclosed on display from behind the user. This problem PS™ 88100 •« »* according to * fourth embodiment of the 

occurs because security is not provided in the physical space P resent invention, 

surrounding the user and the apparatus even if security is 45 FIG. 7 is a block diagram of the sequrity apparatus 

executed inside the apparatus electrically. Especially, this according to a fifth embodiment of the present invention, 

problem happens when the service is supplied to the user in FIG. 8 is a flow chart of processing of the service supply 

public place ia which other persons are not excluded. permission section according to the fifth embodiment of the 

SUMMARY OF THE INVENTION preSCflt tovcntlaB ' 

Tr « an ^ a. - , ■ • FIG. 9 is a flow chart of processing of the service supply 

it is an object or the present invention to provide a • • . . , . 

^ t - nr arw<t * , ^ . * , % " permission section according to a sixth embodiment of the 

security apparatus and method to supply a service to the user ™ emt 

while safety and reliability for the service are kept. * . £ 

According to the present invention, there is provided a JS^l^ ? "f.* 0 ™**** 

security apparatus for a device supplying a service to a user. 55 S^^SJT^ * OTbodiroeDt of 
comprising: person discrimination means for recognizing a 

user requesting the service; use situation decision means for HG ' 11 is 8 block of sequrity apparatus 

deciding whether the user is under a situation to use the according to a eighth embodiment of the present invention, 

service; infringement situation decision means for detecting ^ * s a ^ ow cnart °* p^ocjssing of the service supply 

whether a non-user intrudes into a use area the service to 60 P e ™ ss i° fl section according to the eighth embodiment of 

decide whether of the service is infringed; service control lhe F^ 61 * invention. 

means for supplying the service to the user in case said HG. 13 is a schematic diagram of detection of visual lines 

person discrimination means recognizes the user, and for of the user in the image acceding to the eighth embodiment 

controlling a supply of the service if said use situation of the present invention. 

decision means decides the user is not under the situation to 65 FIG. 14 is a flow chart of processing of the service supply 

use the service or if said infringement situation decision permission section according to a ninth embodiment of the 

means decides that the security of the service is infringed. present invention. 
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FIG. 15 is a schematic diagram of detection of visual lines another person, the supply of the service iaterupts until the 
of the non-user in the image according to the ninth embodi- infringement is relieved. The service supply section 3 sup- 

ment of the present invention. plies various kinds of service under the control of the service 

FIG. 16 is a flow chart of processing of the service supply supply permission section 2. For sample, an uninhabited 

permission section according to a tenth embodiment of the 5 ^^TM (automatic teller's machine), a portable terminal, or a 

present invention. computer used by several persons may comprise the service 

FIG. 17 is a schematic diagram of supply method for the supply section 3. 

security apparatus by software according to the present HG. 2 is a block diagram of the tituation detection section 

invention. 1° ^ s H&ag e is used as a way to discriminate the 

i° user, detecting the use situation, or detect an infringement 

DETAILED DESCRIPTION OF THE situation. The situation detection section 1 is comprised of 

PREFERRED EMBODIMENTS an image input section 11. a person area detection section 12. 

The embodiments of the present invention are described a P^ 50 ? comparison section 13, a use situation decision 

below with reference to the drawings. FIG. 1 is a block secuon 14 and an infringement sitiiation decision section 15. 

diagram of the security apparatus according to the present 15 ^ ™P ut section 11 ^P* 215 an Sffla S c throu S h a TV 

invention. The security apparatus of the present invention is caraera - ^ TV camera « set » &e service use «a for 

comprised of a situation detection section 1, a service supply continuously observing the user to be supplied the service, 

permission section 2 and a service supply section 3. The The person area detection section 12 analyzes images input 

situation detection section 1 discriminates the user in b ^ te i*P«t section 11 and monitors the secure area 

service-use area, detects whether the user is under a situation 20 ftQm ^ ""P™* For example a difference between the 

to use the service (use situation) and detects whether a iD P ut 211(1 background imaf.e (uninhabited area) indi- 

non-user infringes the security of the user (infringement cates a ^ $on m seC1 ^ a^a- The person area 

situation). The detection result of the use situation and the detection section 12 outputs person detection information 

infringement situation is outputted to the service supply D1 w ^ en * e P 6 ™** present is detected and outputs non- 

permission section 2 as person information. In this place, the 25 detectioD information Dl when ihe person present is not 

situation that the user can utilize the service represents that detected. 

the user is confirmed to be permitted the supply of the The person comparison section 13 compares the person 

service and the user continuously utilizes the service. A present area image with the content of a person comparison 

situation that the user continuously utilizes the service 3Q dictionary to specify which person is present in (he secure 

represents a status that the user actually interacts with the area image. The person comparison dictionary previously 

service application (the user inputs operation to system or stores at least comparison information of persons permitted 

the user is supplied output from the system). A status of to use the service. The person comparison section 13 outputs 

non-interaction within predetermined time indicates person discrimination information D2 representing whether 

thinking, chattering, looking away. Another status indicates 35 the person present coincides with a person permitted to use 

when the user moves away from service use-space within a the service. If the person present coincides with a person 

predetermined time. permitted to use the service in the person comparison 

The situation that a non-user infringes the security of the dictionary, the person present is .registered as the user who 

user represents that a person other than the confirmed user 0311 ^ e su PPlied the service. 

intentionally or accidentally uses the service while the ^ The use situation decision section 14 detects whether the 

service is supplied to the confirmed user. The other person user is under a situation capable of using the service. After 

can then interact with the operation or information to be the user of the person area is registered and if the registered 

disclosed only to the confirmed user. When the user interacts ' user is now included in the person discrimination informa- 

with the service application of the system through a display, tion D2. the user is decided to be capable of using the 
the other person looks on the display behind the user. 45 service. The use situation decison section 14 outputs use 

Otherwise, the other person intends to interact with the situation information D3 representing whether the user is 

service application in the user's absence within the prede- capable of using the service. 

termined tiinc * * The infringement situation decision section 15 detects 

In order to detect each situation, the situation detection whether a non-user infringes the security of the service. If a 
section 1 monitors service-use area using image ,or sound. 50 person other than the user utiizing the service now is 

Especially, the use situation of the user and the infringement included in the person discrimination information D2, the 

situation of the security are detected by the image (person infringement situation decision faction 15 decides that the 

detection/direction of face detection/person confirmation) security of the service is infringed, and outputs infringement 

and the sound (person detection/person specification/ situation information D4 indicates a breach of security. The 
chattering detection). Monitoring may be wireless. 55 person detection/non-detection information Dl, the person 

The service supply permission section 2 controls startf discrimmation information D2. the use situation information 

interruption/interruption and cancellation/end of the service D3. the infringement situation information D4 are called 

for the service supply section 3 according to the detection person information. The person information is read out by 

result of the situation detection section 1. In this case, the ft e service .supply permission section 2 (step SI in FIG. 3 
service supply permission section 2 starts the supply of the 60 explained afterward) and referred at steps S3, S4, SS, Sll in 

service when the user is specified in the service-use area. HG. 3. 

The service supply permission section 2 controls the supply FIG. 3 is a flow chart of practising of ihe service supply 

of the service in at least one of the cases where the user does permission section 2. As mentioned above, the situation 

not utilize the service or where the security of the user is detection section 1 discriminate? the user in the service-use 
infringed. Especially when the user does not utilize the 65 area and monitors the use situation and the infringement 

service during the supply of the service, the supply of the situation using the image inputted from the TV camera. The 

service finishes. If the security of the user is infringed by person information as detection Jesuit of the situation detec- 
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tion section 1 is outputted to the service supply permission Next, additional embodiments of die present invention 
section 2. By using the person information, the service will be explained. 

supply permission section 2 executes a security controlling (a) ease of monopoly of tenninal/improvernent of corn- 
process to the service supply section 3. Erst, the service mon U se of terminal (second embodiment) 
supply permission section 2 obtains the person information 5 ^ me & st embodiment, anoto person can not use the 
from the situation detection section 1 (step SI). When the taminal the U5cr is m fte servfce ^ 
supply of the service begins ("No" at step S2), the service terminal ^^ok, the other peisoa must wait to use the 
supply permission section 2 monitors the security areain the same tennifla! ^ ^ ^ USffl . Mshes ^ sha[t ^ fte 
servfce use area according to the person detection/non- &st ^^^^ use by a specified nser foI me teiminid 
detection itformation Dl (step S3). If a person area is not 10 aaMu use . M ^ n 
detected ('No" at step S3), processing of the service supply T . ' .. * . . .„ ^ , 
permission section 2 is returned to ftep SI. On the ottter J» a s « on ^inbodjxnent. »«« the .Situation for the user 
hand, if a person is detected ("Yes" at step S3), the service ■*5*" U ? of " S10 « the "^J* d 2 DCted * fo ? * e 
supply premission section 2 decides whether the person is f f »™ far * e «» ^PP 1 ? ^ me *™« 
perrnitted as a user (step S4). As a result, in case thVperson is » rescrv ? d (f™"£«>)«d D A ew user c f *? ^H*"* 
is not permitted as a user ("No" at step S4), the service * e semce *"» * e J**™* As a "?ult, d,mn S the 
supply permission section 2 rejects the supply of the service of * e sw y ice " a f*" 6 "' "™» » "Pi*** » 
(step S5) and returns processing to step SI. In case the anothcr , ma A me W" 1 *«* » «• 
person is permitted as a user ("Yes" at step S4). the service tenmnaLthe reserved semce is supplied to the ongmal user 
supply permission section 2 registers the person as the user 20 ^ .^ffore while each jervice is not completed 
(step S6). After that, the use situation of the person is ""P***"* «ers « be supplied each service from 
continuously monitored. The service supply permission sec- * e . same termtnal one another - Additionally, if the 
tion 2 begins the supply of the service (step S7) and returns "W** user . does not . f me back to * e same temdmL me 
the processing to SI reserved service is finished and memory area corresponding 

il The above process steps (step SI-S7) are the same as * ^^TTlTtl^^r^?*"^ 

S ncirrnalcoimiTLtfonofse^ nG. 4 is a flow chart of proce«mg of the service supp y 

begming the supply of the service). In this place, after the P™° n »W according to .he second embodiment. In 

C: supply of the service is begun, the registered-user exists in ^ S ^ assume ^aroputer networkw commonly used 

ftl me service-use area and the registeTed person must be user ?„ *** 3 P 1 ****? rf l™?* ^connected to the 

£ detected at step SI. After the supptyof the service is begun, 30 a sec f * f™«ion. In FIG. 4, steps SIS. 

S the processing is returned to step S8 through steps SI and S16, S17. S18 are newly added in comparison with FIG. 3. 

|%: Each user is confirmed respectively and supplied the service 

j ' , . . , through a respective terminal (step SI, S2). 

— The service supply permission section 2 decides whether Trl „ .w » 

« meregisteredpJoVisusmgthese^^ * T T 1 * Ser * ! 

situatbn infoimition D3 (step S8). As a resllt. if the 35 USU1 8 * f™ 81 A and leaves fram the terminal A without 

„f »k» " j I ■? . * ^ , completing the work. The terminal A detects that the user A 

^ TJt L g f ^ - fW , CTan3 ^ f * °<* "sing *e service ("No" at .step S8) and that the supply 

W T^^Z^^^**^ 0 ""?'*"??* 0 - of is not completec (Wat step S15). T^e 

?■ "i tw» ^«,"ot^ot:^ ^ *u • « j / \ 01 ^ j 40 (reserved) in a memory area of ihe terminal and the supply 

the registration or the user is cancelled (step SIC} and the * - i • • ^ ^ . ^- ^ rTT-. , . . 

+ P1 . v»«F J wc of the temuial is interrupted (step S17). In this case, the 
processing returns to step SI in order to register a new user. ^ +u r ^ v . r * y . " ' , 
^ r *^ situation for the user not usicg the service is detected 
On the other hand, the service supply permission section according to the use situation information D3- Completion 
2 decides whether a non-user infringes the security of the situation of the supply of the service is decided by comple- 
ter according to the infringement situation information D4 45 tion notice sent by the user. Next, a user B is confirmed by 
(step SU). As a result, if the situation that a non-user the teimnalAandwcrics through the terminal A. Assume that 
infringes the security is detected, for example, an other when the user A comes back to the terminal A, the user B 
person watches the display behind the user during the supply goes away from the terminal A The terminal A newly 
of the service ("Yes" at step Sll), the service supply reserves the results of the service for the user B (steps S8 
permission section 2 temporarily interrupts the service to 50 SIS. S16. S17) and begins the supply of the reserved service 
maintain . security untill the security is not infringed (step to the user A (steps S8, SU. S13, S14). The memory area to 

512) . If a situation that the security is not infringed is store the result of the service is commonly used by each 
detected during the interruption of the service ("Yes" at step terminal on the network Therefore, if the user B is con- 

513) . the service supply permission section 2 cancels the firmed by a terminal B (steps S4 S6, S7), the reserved result 
interruption of the service and begins the supply of the 55 0 f the service for the user Bis si pplied through the terminal 
service (step S14). B (steps S8, SU, S13, S14), On the other hand, if the user 

In this way, in case the user permitted the service is A does not come back to the taminal a during a predeter- 

specified in the service use area, the supply of the service is mined time ("Yes" at step S18). the terminal A finishes the 

begun. During the supply of the service, if the user is not service of the user A immediately and relieves the memory 
using the service or if the security is breached, the supply of 60 area corresponding to the service result of the user A (steps 

the service finishes or interrupts. Accordingly, the security is S9, S10). In this way, a situation of storing many service 

maintained during all periods of begining/supplyin^end of results in memory area is preven ted. The predetermined time 

the service. For example, in case the registered user is as decison standard at step S18 is freely set by the kind of 

confirmed and supplied money through an Automatic Teller the service. 

Machine, or the iuser operates a handy terminal by watching 65 As mentioned above, in tht; second embodiment, the 

secret information on a display. &e security of the user is supply of the service for each us-;r is reserved untiil the work 

effectively protected according to the present invention. of each user is completed. In short a right service-use 
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generated by confirming the user is assured untill the user's fifth embodiment In FIG. 7. an interactive control section 4 
work is completed The user B does not have to wait until! is a new section in comparison with FIG. 1, FIG. 8 is a flow 
the user A finishes using the tenninaL By changing the user chart of processing of the service supply permission section 
A for the user B, the user B can use the terminal without 2 according to the fifth embodiment la FIG. 8, steps S20. 
infmge^enrttii\flie security of the user A. 5 S21 are new added steps in comp arison with FIG. 3, 

xtention fo\ unspecified user (third embodiment) The interactive control section 4 receives an^fiScafibn of 

^grste^odiment, in order to be supplied the ^ c usei for warning of the iimiagementan|^ntroUstiie 

service, theTIsermust be confirmed as a registered person. supply of the service according to the k(UcatioESI3i£jfcd 
In short if the user is not previously registered, he can not of the indication is contmuatioti/interruptionVeDd of the 
be supplied the service. However, mere is a service to be 10 service. The user selects the kind of the indication by. for 
used by many and unspecified persons. Therefore, in a third example, using a keyboard. In FIG. 8. during the supply of 
embodiment, the service is supplied to an unspecified person me service, if the situation detection section 1 detects 
without confirmation. At that time, a comparison dictionary infringement by a non-user ("Yes 1 * at step Sll), the service 
of theunspecifledpersonis generated and registered. During supply permission section 2 sends the warning to the user 
the supply of the service, the user (the unspecified person) 15 (step S20). After sending the waning, the interactive control 
is discriminated from other users by referring to the com- section 4 receives an indication cf the user (step S21), the 
parison dictionary. Therefore, the service is supplied to service supply permission section 2 controlls the supply of 
many and unspecified persons while the security is main- the service according to the indication, 
tained. If the user indicates continuation of the service 

FIG. 5 is a flow chart of processing of the service supply 20 ("CONnNUAFTON" at step S21), the interactive control 
permission section 2 according to the third embodiment In section 4 outputs the indication of continuation to the service 
FIG. 5. step S19 is newly added instead of steps S4, S5 in supply permission section 2. The service supply section 3 
comparison with FIG. 3. When the supply of the service is continues to supply the service according to the indication of 
begun ("No* at step S2), the service supply permission continuation. 

section 2 detects the person in the service use area according 25 If the user indicates interruption of the service 
to the person detection/non-detection information Dl (step ("ETOSRRUFnON'* at step S2:L), the interactive control 
S3). If the person is detected ('Yes" at step S3), the service section 4 outputs the indication ol interruption to the service 
supply permission section 2 generates the person compari- supply permission section 2. This service supply section 3 
J! son dictionary (step S19) and registers the person as a new interrupts the supply of the service according to the indica- 

jj user (step S6). After that the use situation of the new user 30 tion of foejntouption. 

M is continuously monitored, and the service supply permis- If the^er^nSJtates end of the service ("END" at step 

L ^ on sectioa 2 teg&s to supply the service (step S7). In this S21). the mterractivec>ntrol section 4 outputs the indication 

13 way. the service is suppUed to an unspecified person without of eadNo the servfee supply permission section 2, The 

ffi confirmation and the person comparison dictionary is gen- service supply section 3 finishes the supply of the service. 

U crated to be registered. Therefore, a new user except for the & this way. when the user receives the warning of 

f ;■ pre-registered user is able to be supplied the service. In this infringement the user can select continuation/interruption/ 

W case, during the supply of the service for the new user, he is end of service by using interactive function. As a result if 

fg discriminated from another person by referring the person the user works through equipment in an oflce. whenever an 

y. comparison dictionary in order to maintain the security. ^ office colleague (person not necessary for security) stands 

(c) Warning of infringement (fourth embodiment) behind the user, interruption of the work is excluded. In 
In the fourth embodiment, when an infringement situation short, a change for the worse of u j-e-feeling of the equipment 

of the security is detected, a warning is sent to the usefTTn^N is avoided. Otherwise, when using the equipment in a public 
this jai^tte service supply permission section 2j^^oUsy place, interruption of the service is indicated by the user for 

tegim'g^nd end of the service and does noT«BscIIfe 45 the warning of a non-user standing behind the user. 
MeSop^Qn^d cancellation of the service in case of In the fifth embodiment the uner can select the indication 
infringement of the security. to keep security in case of infringement of the security. After 

FIG. 6 is a flow chart of processing of the -service supply selecting the indication, the apparatus (equipment) can auto- 
permission section 2 according to the fourth embodiment In matically keep the security of die user. In this case, if the 
HG. 6, step S20 is a newly added step instead of step S12 50 warning of the infringement is outputted as an image, the 
in comparison with FIG. 3, First, when the situation detec- user understands circumstances itround the user by watching 
tion section 1 detects the infringement situation of non-user the image (the user and non-user are included). The user can 
("Yes" at step SH). the service supply permission section 2 select the indication (continuation/interruption/end of the 
sends a warning of the infringement to the user (step S20). service) according to the circumstances. 
Concretely speaking, the warning is outputted through an 55 (e) Default action of negketing the warning (sixth 
image (warning sentence/mark, controlling of color/ embodiment) 

brightness/blink) or sound (speaking of warning sentence/ the sixth embodiment in case the indication is not 

warning buzzer). When the user receives the warning, it is selected by the user within predetermined time after gener- 
possible for the user to protect his security by, for example, ating the warning, the appara:us automatically executes 
rurnfog off a power surely of to^ ^ predetermined (default) controlling of the supply of the 

of the terminal. service. This predetermined controlling is freely set by the 

(d) Interactive controlling in case of warning (fifth user, 
embodiment) pjQ 9 ^ a flow of p r0c£SS u,g 0 f fo e service supply 

In the fifth embodiment by using input means (key board) permission section 2. In HG. S>, steps S2D.S21.S22.S23.S24 
and output means (display), the supply of the service is 65 are added steps in comparison with FIG. 3. The construction 
controlled by the user's operation for the warning. FIG. 7 is of the security apparatus of the i tfth embodiment is the same 
a block diagram of the security apparatus according to the as in HG 7. In HG. 9, during the supply of the service, if 
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the situation detection section 1 detects an infringement document While the user creates the document whose 

situation ("Yes" at step Sll). the service supply permission security degree is low, the security degree of the document 

section 2 sends the warning to the user (step S20). After fee grows by adding another important document. In this way, 

warning, the interactive control section 4 receives indication the security degree is dynamically changed according to the 

of the user. In this case, if no indication of the user is 5 i-fad 0 f the seraceyinforroation. 

received above predetermined time (step S22. S23). the ^ ^ seventh cmbodiment . ^ sccurity degree is 

attractive control section 4 outputefce^urport of no ch db unjt of the ser¥ ice/Mcrmation. FIG. 10 is a flow 

UKUcation to the service supply pe^s^section 2 The chart of si ofthe service supply permission section 

servrce supply passion se*on£c«r^^ 2 according to the seventh embodiment. In FIG. 10. step S25 

the service accoidingto a pre^terrWelS3!6ol (step S24). 10 added betw^n Sll and S12 is a Afferent step in comparison 

For sample if continuance se^e is set as the with nG.3.Duringthesupply of the service if the situation 

pxedeterrmned contro CWNnNUAnON-- at step S24). detection section \ ^ "infringement of the security 

fte service supply ection 3 conbnues to supply fce semce^ c<Yes „ «, U) ^ ^ J permission section 2 

If lntecruption of the service is set as the predetermined V . * ' , _ . ^ _ j- . *v. 

/t ^xTrrT-TD t> t rrrrrf r\x^i * * *l ■ i controlls the supply of the service according to the security 

control ( INTERRUPTION^ at step S24), the service supply „ , _ . . *£ %L j * /• * / * * 

- * . . * 1 • « j f!L 15 degree set by the kmd of service/ififormatioEi (step S25). In 

2££ VSTTS! 7?* ? J" T?' * thifLse. itLy be possible that the security degree is set to 

AT *r*? y T^! * *%T y example, when anomex person watches me dispky behind 

the service. On the other hand, if the indication of the user f , „„ . t . , ■ . . . 

is received within the predeterroined time fW at step 20 

S22).mem t eractivecon^ 2 ° ^ closed) and the WP window continuously opened 

(step S21) and the service supply permission section 2 As for setUll S the security degree, for example, the user 

W contrails the supply of the service according to the indica- sets disclosure-possible at the hsad portion of mail to be 

m tion. In this case, if the user indicates continuation of the seilt A win<iow lowing the mail tool does not react to the 

m service (XOiraNUAnON" at step S21). the interactive ~ i^gement of the security. As for both sending side or 

S control section 4 outputs the continuation indication to the recei ving side, the mail is able to remain disclosing. 

service supply permission section 2. The service supply ^ tne user indicates a strict secret at the header of a 

section 3 continues to supply the service. If the user indi- document, the WP window to display the document reacts to 

cates interruption ofthe service ("JNTERRUPTION'* at step tne infringement of security. For example, the sequrity 

S21), the interactive control section 4 outputs the interrup- ^ degree for creating the document is dynamically changed by 

tion indication to the service supply permission section 2. adding other important document. In this way, even if the 

The service supply section 3 interrupts the supply of the service is supplied to the same object, the supply of the 

% service. If the user indicates end ofthe service ("END" at ■ service is interrupted or continue according to the content 

step S21). the interactive control section 4 outputs the end ff or example,, regular document or strict secret document), 

indication to the service supply permission section 2. Hie 35 (g) detection of visual fine 1 [eighth embodiment) 

service supply section 3 finishes the supply of the service. in the above embodiments, when another person watches 

In this way, if the user neglects a warning of infringement the display behind the user seating a document, the 

the security of the user is kept according to a predetermined infringement of security of the user is detected However, it 

control (action of default). Otherwise, it is possible to is not necessary to pay attention to the other person, but to 

control the supply of the service different form the prede- 40 pay attention to the visual line (or line of sight) of the user, 

termined control for some reasons by a user's indication in For example, when the visual line of the user leaves the 

case of the warning. For example, in an office, as for the screen of the display, the window may be closed. In the 

warning generated whenever a colleague stands behind the eighth embodiment by detecting a visual line of the user as 

user operating the equipment, the user does not have to use situation, the supply of the service is interrupted when 

indicate continuation of the service. The predetermined 45 the visual line leaves the screen. 

control is already set as continuation of the service. In this FIG. 11 is a block diagram of the situation detection 
case, the user can continue the operation while neglecting section 1 according to the eighth emrx>diment A visual line 
the warning. In case the user operates the equipment in a detection section 16 is an added section in comparison with 
public place, the predetermined control is set as interruption HG. 2. The visual line detection section 16 detects move* 
of the service. In mis case, security of the user for the 50 ment of the visual line of the user in case the user 
equipment 1$ protected while he neglects the warning. (registered) is included in the j«rson discrimination inform 
(f) Security degree by unit of service/information (seventh mation 02. For example, the face area of the user is 
embodiment) extracted from the image and the eye area is extracted from 
In the above embodiments, predetermined security is the face area. The visual line is pursued according to the eye 
activated for service or information. However, security 55 area. In this case, if a direction of the visual line is toward 
degree is actually different by unit of service/information. If a predetermined area (visual area of the screen), an indica- 
a fixed security degree is activated, useless interruption of tion is outputted If the direction of the visual line is not 
the service is often executed. This happens in case of plural toward the predeterrnined area, .another indication is output- 
workers using information equipment. Accordingly, it is ted. These indications represent visual line information D5. 
desired that the security is activated according to the kind of 60 In this way, in the eighth embodiment the visual line 
the service/information. For example, assume that electric information D5 is outputted to the service supply permission 
mail is transmitted using a PC (personal computer) or a section 2 as one of the person icformation. HG. 12 is a flow 
document is created using WP (word processor). Plural chat of processing of me service supply permission section 
services are then supplied on the display of the equipment 2 according to the eighth embothment. In FIG. 12, step S26 
The electric mail represents a private letter and its security 65 is added instead of step Sll in comparison with HG. 3. 
degree is high. On the other hand, the security degree of the During the supply of the service, assume that the situation 
document by WP differs according to the kind of the detection section 1 detects that the user is under a situation 
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to be supplied the service ("Yes" at step S8). In this case, the (i) Detection of visual line 3 (tenth embodiment) 

visual line detection section 16 keeps watch for the move- & the tenth embodiment, detection of the visual line of the 

ment of the visual line of the user. If the direction of the ma; an d the non-user contrails the supply of the service, 

visual line departs from the predetermined area of the Y1G, 16 is a Sow chart of processing of the service supply 

display ("Yes" at step S26). the situation detection section 1 5 permission section 2 according to the tenth embodiment. In 

outputs the visual line information DS to the service supply HG. 16, step S26 and step S27 arc added steps in compari- 

permission section 2. The service supply permission section §on with HG. 3. During the supply of the service, assume 

2 controls the service supply section 3 to interrupt the supply that the situation detection section 1 detects that the user is 

of the service (step S12). HG. 13 is a schematic diagram of a situation to be supplied toe service ("Yes" at step 

.detection of the visual lines of the user in die image 10 $$) t ^ this case, the visual line detection section 16 detects 

according to the eighth embodiment As shown in FIG. 13, movement of the visual line of the user. If the visual line of 

the visual lines of the user are detected from position of the user departs from the predetermined area, for example, 

black eyes of the user's face in the image. the user turns his eyes away from the display ( M Yes M at step 

In this way, in the eighth embodiment, detection of the S26), the situation detection section 1 outputs the visual line 

visual line of the user controlls the supply of the service. 15 information D5 to the service supply permission section 2. 

Therefore, when another person walks behind the user, Then, the service supply permission section 2 controlls the 

security of the survice is kept only if the visual line of the service supply section 3 to interrupt the supply of the service 

user departs from the display. (step S12), 

(h) Detection of visual line 2 (ninth embodiment) On the other hand, assume thar; the direction of the user 

In the eighth embodiment, a direction of the visualline of 20 P° ints t0 ^ predetermined area of the display (the user does 

the user controlls the supply of the service. However, if aot tani his eyes away from the display) ("No" at step S26) 

another person looks furtively at the display behind the user* ail< * ^ situation detection section 1 detects that the non-u ser 

the supply of the service can not be controlled. If the user infringes the service of the user ( 'Yes" at step Sll). In this 

turns his eyes away from the display whenever the other „ case - tne visual line detection section 16 keeps watch for 

person walks behind the user, the supply of the service is movement of the visual line of the non-user. If the direction 

interrupted to every time. Therefore, in the ninth ofthe visual line of the non-user points to the predetermined 

embodiment, detection of the visual line of the non-user 3168 °f toe display (the non-user watches the display behind 

controlls the supply of the service. &e user) ("No w at step S27), the situation detection section 

HG, 14 is a Sow chart of processing of the service supply 3Q 1 mt P nts visual ^ formation to the service supply 

permission section 2 according to the ninth embodiment In Passion section 2. The service supply permission section 

HG. 14. step S27 is added step between step Sll and step 2 controlls the service supply section 3 to interrupt the 

S12 in comparison with HG. 3. In this case, the visual line su ^ of Ae service < stc P S12 >- 

detection section 16 detects the visual line of the non-user. ^ wav < k th c tent h embodiment detection of the 

In case the person is not discriminated by referring to the 35 visual line of the user and fee noa-user controlls the supply 

person discrimination information D2 (the person is decided °* * e service. Therefore, the security of the service fox the 

to be a non-user), the movement of the visual line of the user is strengthened. 

non-user is detected. In the same way as in the eighth In the eighth, ninth, tenth embodiments, the visual line of 

embodiment the face area is extracted from die image the user or the non-user is detected. However, by detecting 

including the non-user and the eye area is extracted from the 40 a direction of a face of the user or non-user, the supply of the 

face area. The visual line of the eye area is pursued. If a service may be controlled. In this case, as for the user, if the 

direction of the visual line of the non-user points to a direction of the user* s face does not point to the display, the 

predetermined area (visible area of the display), an indica- supply of the service is interrupted. As for the non-user, if 

tkm is outputted. If the direction of the visual line of the the direction of the non-user*s face points to the display, the 

non-user does not point to the predetermined area, another 45 supply of the service is interrupted As for the method for 

indication is outputted. This purport is represented as the detecting the direction of the fac;, the face area is extracted 

visual line MormationDS. During the supply of the service. from the image including the user or the non-user. The 

assume that the situation detection section 1 detects that the direction of the face is pursued according to the face area, 

non-user infringes the security of the nonmser ("Yes" at step In the above embodiments, the image is used for detecting 

Sll). In this case, the visual line detection section 16 keeps 50 the person information. However, sound or wireless may be 

watch for the move of the visual line of the user. If fee used for detecting the person iaforroation. In case of the 

direction of the visual line of the non-user points to the sound , the user is specified by fee user's voice existing in 

predetermined area, for example, the non-user watches the the service-use area. If the usa-'s voice is continuously 

display behind the user ("No" at step S27), die situation detected, it is decided that the user is under a situation to be 

detection section 1 outputs the visual line information to the 55 supplied the service. If a voice other than the user's is 

service supply permission section 2. The service supply detected, it is decided that the siscurity of the service of the 

permission section 2 controlls the service supply section 3 to user is infringed In case of the wireless, the user is specified 

interrupt the supply of the service. HG. 15 is a schematic by detecting ID information of a wireless card. The user 

diagramof detection of the visual lines of the non-user in the previously brings the wireless card with him. If the 3D 

image according to the ninth embodiment As shown in HG. » information of the user's wiruless card is continuously 

15, the visual lines of the non-user are detected from detected, it is decided that theu£;er is under a situation to be 

position of black eyes of the non-user's face in the image. supplied the service. If ID infornation other than the user's 

In this way, in the ninth embodiment the detection of the wireless card is detected, it is decided that the sequrity of the 

visual line of the non-user controlls the supply of the service. service of the user is Mringed. 

Therefore, in case the non-user looks furtively at the display 65 HG. 17 is a schematic diagram showing a method of 

behind the user, the security of the service for the user is supplying a software for the present invention. A series of 

^P*- processings of the service supply permission section 2 



